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1.   IMAGE  PROCESSING  PROPOSAL 

Picture  Processing  Strategy 

Strategy  is  proposed  to  permit  personnel  (not  trained  in 
computer  technology)  to  input  scanning  instructions  directly  to  the 
image  processing  system.   Cartoons  of  sample  pictures  are  used  to  define 
a  training  set.   Continues  work  presently  underway.    (Section  2) 

Plane  Parallel  Processing  Strategies 

It  is  our  intention  here  to  devise,  implement  and  evaluate 
parallel  processing  strategies  appropriate  to  the  Pattern  Articulation 
Unit  of  the  I Iliac  III  computer.   Procedures  to  be  investigated  are: 
1)  shape  determination  from  scenes  of  shaded  objects,  2)  fast  articulation 
of  (quasi)  line-like  images,  and  3)  detection  of  differential  changes  in 
scene  representation  with  motion.  (Section  3) 

List  Processing  in  Taxicrinic  Processors 

The  appropriateness  of  the  Taxicrinic  Processors  of  Illiac  III 
as  a  vehicle  for  the  implementation  of  list  processing  languages  will  be 
evaluated.   The  class  of  features  peculiar  to  this  processor  include: 
virtual  addressing,  operand  and  pointer  stacks,  hardware  implementation 
of  available  space,  list  instructions  and  the  ability  to  execute  a  class 
of  procedure  calls  entirely  in  hardware.  (Section  k) 

Applications 

In  depth  investigation  will  be  made  of  Scanning  Electron  Microscope 
(SEM)  photographs  of  polycrystalline  and  biological  materials  as  a  variant 
of  scene  analysis.   Use  of  multiple  modes  (emissive,  conductive  and  luminescent) 
of  the  SEM  to  give  multi-spectral  description  of  scene  will  be  probed. 
Possible  application  to  radiation  damage  and  materials  research  are  contem- 
plated.  Liaison  with  the  AEC  supported  Materials  Research  Laboratory  (MRL) 
at  Illinois  will  be  established.  (Section  5) 


2.   PICTURE  PROCESSING  STRATEGY 

2.1  Research  Goal 

The  object  of  this  research  is  the  development  of  a  strategy 
to  permit  personnel  (not  trained  in  computer  technology)  to  input  scanning 
instructions  directly  to  the  image  processing  system. 

Conceptually  we  can  view  this  goal  as  a  necessary  first  step 
toward  the  eventual  realization  of  a  scanning  robot:  a  machine  which  given  a 
limited  visual  environment,  and  appropriate  training,  can  be  said  to 
recognize  and  describe  the  objects  of  its  environment  with  an  aptitude 
approaching  that  of  a  trained  scanning  technician.   To  approach  this  ideal, 
a  conventional  computer-based  image  processing  system  must  be  augmented  by 
several  facilities:   (l)   Given  an  appropriately  structured  learning  experience, 
the  augmented  system  must  formulate  its  own  image  processing  strategy. 
(2)   This  strategy,  continually  guided  by  evaluative  feedback,  must  provide 
command  and  control  to  the  image  processing  system,  and  so  dictates  the 
frame,  orientation,  field  of  view,  magnification,  etc.  of  current  attention. 
And  (3)  the  system  must  provide  as  output  a  model  of  the  environment,  rather 
than  merely  a  frame-by- frame  image  description.   This  proposal  describes 
strategies,  existing  image  processing  equipment  and  experiments  appropriate 
to  explore  the  potential  of  a  scanning  robot. 

We  observe  the  computer-based  scanning  system  as  conceived  here 
is  not  intended  to  produce  polished  code  for  the  recognition  task  at  hand. 
Rather  the  primary  role  of  the  image  processing  package  is  to  establish 
feasibility  of  the  scanning  task.   As  attached  to  a  Scanning  Electron 
:ope  (SEM),  for  example,  this  system  might  be  directed  to  determine 
tribution  of  particle  parameters,  or  for  polycrystalline  materials 
,imate  for  each  crystal  subpopulation  the  corresponding  surface-to- 
ratio.   Alternate  graded  sets  of  scanning  exercises  can  be  readily 
>ther  areas  of  application.   The  image  processing  requirements 
g,  however,  typically  embrace  both  ends  of  the  task  spectrum: 

scanning  task  is  variable  and  short-lived — any  quick  results 
.on  of  particle  parameters)  would  be  gratefully  received;  or 
-  is  extensive  and  warrants  several  man  years  of  effort — 


were  feasibility  of  the  scanning  task  once  demonstrated.   It  is  for  these 
two  task  situations,  viewed  exclusively  as  feasibility  experiments,  that 
the  design  of  the  automated  image  analysis  system  is  directed. 

2. 2  Required  Faculties 

These  faculties  must  be  provided  the  image  processing  package: 

(1)  A  programming  language  to  describe  the  scanning  task  must 
be  developed.   It  must  be  possible  to  specify  the  image 
enhancement,  pattern  articulation,  syntactic  analysis 
(where  available)  and  model  construction  aspects  of  the 
scanning  task. 

(2)  The  learning  experience ,  defined  in  the  terms  of  input 
cartoons, accompanied  by  interactive  labelling  of  edges  and 
regions,  etc.,  must  be  carefully  prescribed.   The  automaton 
in  this  way  is  provided  with  primitive  features  to  extract 
and  label  from  test  images  so  that  training  sets  are 
adequately  defined. 

(3)  The  formulation  of  queries  addressed  to  the  scanning  super- 
visor must  be  possible.   For  an  intelligent  automaton  must 
recognize  those  occasions  when  its  experience  is  insufficient 
to  cope  with  new  imagery.   In  this  interactive  manner  the 
scanning  robot  can  be  expected  to  converge  on  useful 
strategies.   And  finally, 

(k)      Monitoring  capabilities  must  be  provided  to  evaluate  the 
performance  of  the  automaton  and  to  allow  the  observer  to 
modify  the  scanning  procedure.   Display  facilities  must 
include  introspective  report  generation  capabilities 
sufficient  to  make  transparent  the  invoked  current  strategy. 
Should  the  application  later  warrant  refined  design  of  an 
optimal  scanning  strategy,  it  should  then  be  possible  to 
transfer  the  accumulated  experience  of  the  prototype 
scanning  system. 


2. 3  Illustration:  Use  of  a  Cartoon  to  Define  Preprocessing  Tactics 

Methodologically,  current  pattern  recognition  research  as  applied 
to  image  preprocessing  has  been  negligent  in  several  regards: 

1)  The  scanning  task  to  be  performed  is  often  inadequately 
defined — so  the  learning  task  requires  undue  artificial 
intelligence. 

2)  The  a  priori  probability  distributions  characterizing  the 
signal  families  to  be  distinguished  are  rarely  evaluated 
systematically  during  the  learning  experience. 

3)  Ad  hoc  preprocessing  tactics  are  often  then  performed  on  the 
picture  without  any  evaluation  of  what  optimal  discrimination 
is  inherently  possible. 

h)      An  artificial  distinction  is  made  between  the  discrimination 
of  regional  attributes,  such  as  texture,  and  the  detection 
of  edges.   In  fact  these  techniques  are  complementary:   regions 
of  common  attribute  (e.g.  color,  texture,  etc.)  are  circum- 
scribed by  edges;  and  conversely  boundaries  (edges)  can  be 
inferred  as  the  limiting  case  of  "roadlike"  areas  separating 
regions  of  common  attribute.   To  find  a  cellular  mosaic 
decomposition  of  the  input  picture  in  general  requires  con- 
comitant use  of  both  techniques. 

A  new  strategy  for  cellular  decomposition,  which  largely  circumvents 
above  criticism,  has  recently  been  developed  here  and  is  now  undergoing 
experimental  test.   In  this  scheme  the  human  observer  prepares  the  input 
picture  (as  an  8  x  10  inch  photograph)  and  its  corresponding  cartoon:   an 

overlay  sketch  of  those  edges  and  by  implication,  regions  the  user  wishes 
recognized  by  the  automatic  image  processing  facility.   (See 
guret:  --3;.  Tne  choice  of  line  width  of  a  cartoon  edge  reflects  the 
bainty  in  the  definition  of  the  corresponding  boundary. 

simultaneous  photo  reduction  of  the  picture  and  its  corres- 
: th  care  taken  to  preserve  the  registry  of  the  two  images, 
.':d  is  vacuum  clamped  to  the  film  platten  of  a  S-M-V  film 
Thee*      ;ports,  developed  originally  for  bubble  chamber  data 
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processing,  provide  a  dual  position  film  platten  (to  accommodate  the  very- 
long  aspect  ratio  of  bubble  chamber  film)  and  so  allow  accurate  mechanical 
translation  between  the  picture  and  its  corresponding  cartoon. 

The  cartoon,  of  course,  can  be  easily  tracked  and  serves  to  define 
a  set  of  edges,  which  can  be  then  labelled  by  teletype  intervention.   In 
addition  the  cartoon  implies  a  cellular  mosaic  decomposition  of  the  image 
into  regions  of  common  attribute:   intensity,  color  or  texture — which  in 
turn  can  be  labelled  by  the  human  monitor. 

Accordingly  the  cartoon  provides  an  articulation  of  the  test  image 
a  training  sequence  of  possible  sample  points  on  the  cartoon  lines  (for  edge 
detection)  or  interior  to  each  enclosed  region  (for  texture  discrimination) 
can  be  determined.   For  each  such  sample  point  a  disc-shaped  neighborhood 
centered  about  the  sample  point  is  defined.   The  array  of  gray  scale  values 
at  each  picture  element  within  the  neighborhood  (i.e.  disc  about  the  sampled 
point)  defines  a  primitive  feature  vector.   Pattern  subpopulations ,  so 
identified  by  labelling  of  the  cellular  mosaic,  can  be  used  to  parameterize 
the  pattern  classifier — subject  to  appropriate  signal  detection  theory 
analysis. 

The  pattern  classifier  so  determined  can  then  be  applied  to  similar 
but  previously  untried  pictures.   For  each  such  picture  a  corresponding 
"cartoon",  though  now  with  noise  and  gaps,  is  developed.   After  appropriate 
refinement  this  cartoon  induces  a  cellular  decomposition  of  the  input  image. 

2.U  Formal  Model  for  Picture  Processing 

The  object  of  this  research,  as  asserted  above,  is  contingent  upon 
the  development  of  a  formal  model  for  picture  processing. 

The  model  emphasizes  the  global  aspects  of  processing,  i.e.  some 
sing  is  done  on  the  image  before  it  is  treated  by  this  model.   These 
sses  will  typically  consist  of  parallel  operations  which  can  be 

-ly  applied  to  filter  the  raw  image  for  enhanced  input  to  the  higher 
.;.   Adaptive  filtering  strategies  for  this  purpose,  as 
lal  detection  theory,  are  the  natural  result  of  the  pre- 
iroduced  in  the  previous  section. 


Five  major  subtasks  in  our  framework  for  picture  processing  must 
be  expressible  in  terms  of  the  model: 

•Determination  of  primitive  cells. 

•Determination  of  properties  or  attributes  of  cells. 
•Determination  of  relations  between  adjacent  cells. 
•Extension  to  relations  between  groups  of  cells  or  non- 
adj  acent  cells . 
•Transformations  to  composite  cells  or  cell  merging. 
This  order  is  preserved  in  the  discussion  below. 

2.U.1  Cellular  Mosaic  Decomposition  of  the  Scene 

The  model  attempts  to  develop  the  structure  and  semantics  necessary 
to  treat  a  general  two-dimensional  scene.   To  achieve  this  goal,  we  intro- 
duce the  first  characteristic  of  the  model.   The  picture  is  initially  mapped 
into  a  cellular  mosaic :a  decomposition  of  the  scene  into  regions  bounded  by  simple 
curves  (cells),  which  then  form  primitives  for  further  analysis.   (Figure  h) . 
The  choice  of  primitives  strongly  influences  the  development  and  the  capability 
of  the  model.   Cells  seem  to  be  far  superior  to  primitives  consisting  pri- 
marily of  line-elements,  which  were  used  in  earlier  approaches.   Line-elements 
were  often  chosen  to  attempt  to  obtain  a  model  based  on  one-dimensional 
and  context-free  grammars,  which  are  inadequate  in  many  cases. 

After  the  articulation  of  the  picture  into  cellular  regions,  spatial 
relationships  and  adjacency  relationships  can  be  conveniently  expressed. 
Varying  levels  of  attention  can  be  applied  to  different  regions  by  the 
choice  of  cells. 

The  determination  of  cells  does  not  follow  directly  from  edge  or 
line  detection.   Regional  attributes,  such  as  texture,  are  important  features 
for  cell  selection.   Since  many  choices  for  cells  usually  exist,  strategies 
to  resolve  the  ambiguity  must  be  found.   It  is  our  hypothesis  that  most  two- 
dimensional  patterns  will  be  amenable  to  an  initial  cellular  articulation 
strategy. 

2.U.2  Graph  Representation 

The  second  characteristic  of  the  model  is  the  abstract  relational 
graph  representation  used  at  most  higher  levels  of  the  process.   The  structural 
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model  allows  arbitrary  binary  relations  between  items.   This  facilitates 
recognition  procedures  based  on  graph  transformations. 

An  important  class  of  procedures  forms  objects  in  a  "name- 
independent"  manner,  i.e.  without  having  to  match  each  element  to  an  explicit 
named  prototype.   These  procedures  can  choose  appropriate  graph  transformations 
given  relevant  properties  of  the  relations  between  items. 

This  class  of  procedure  will  be  used  in  the  lower  levels  of 
analysis  in  order  to  increase  the  efficiency  over  a  linguistic  driven 
analyzer. 

2.U.3  Composite  Formation 

The  graph  representation  is  chosen  for  global  level  picture  pro- 
cessing. This  level  of  generality  is  needed  to  express  the  many  different 
and  complex  relationships  among  sub-elements  of  a  picture. 

The  recognition  process  can  then  be  viewed  as  replacement  rules 
operating  on  graphs.   This  process,  which  is  analogous  to  parsing,  is  called 
"composite  formation".   A  replacement  rule  forms  a  composite  element  from  sub- 
elements  of  the  graph  (Figures  5-6).  The  new  graph  formed  by  new  composite  elements 
and  new  relations  involving  these  elements  is  then  the  candidate  for  further 
composite  formation.   For  successful  recognition,  this  process  should  tend 
toward  a  "recognizable  graph",  i.e.  a  graph  which  has  somehow  been  specified 
as  an  acceptable  goal. 

We  visualize  the  structure  of  this  process  as  a  three-dimensional 
tree  where  graphs  form  planes,  or  sections,  at  various  levels.   Composite 
forming  graph  transformations  operate  between  levels.   At  the  highest  level 
is  the  root  of  the  tree  which  represents  the  final  composite  recognizable 
graph . 

2.k.k     Structural  Component 

To  carry  out  the  processing  steps  we  introduce  three  spaces  which 
parameterize  the  operations  of  the  model.   These  are  the  event  space,  the 
structure  space,  and  the  data  base.   One  of  these,  the  structural  ingredient, 
is  discussed  below. 

The  structure  space  provides  the  means  for  defining  the  form  of  an 
input  picture  which  is  capable  of  being  successfully  processed.   Its 
analogy  in  string  processing  is  the  grammar  of  a  language. 
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COMPOSITE  FORMATION 
Composite  cells  (wide  lines)  are  formed  from  the  initial 
cellular  mosaic  (thin  lines)  of  the  picture  in  Figure  5. 

Lower  level  graph  transformations  operate  entirely  within 
the  boundaries  of  the  composite  cells. 

13 


In  order  to  achieve  the  major  steps   for  two-dimensional  picture 
processing,   the   structure   space  must  be   capable   of  representing  hierarchial 
structures   of  elements  with   attributes   and  interrelated  by  binary  relations. 
A  structure   is  built   up  by  applying  set-replacement   rules,   a  generalization 
of  grammatical  productions.      Predicates   are  used  to  specify  when   a  rule  may 
be   applied. 

The  advantages   of  associating  semantic   attributes   as   integral 

2k 

parts  of  grammatical  rules  have  been  treated  by  Knuth   for  the  case  of 

context-free  string  grammars.   Knuth  claims  that  having  attributes  and 
attribute  value  assignment  functions  associated  with  each  rule  allows  the 
constructs  of  the  language  to  be  defined  in  terms  of  their  "immediate 
environment"  and  minimizes  the  interconnection  between  definitions  of 
different  parts  of  the  language.   In  the  same  spirit,  in  the  two-dimensional 
case,  we  also  associate  predicates  with  each  set  replacement  rule  to  specify 
the  necessary  relations  between  elements. 

It  would  appear  that  in  picture  processing  the  appropriate  set- 
replacement  rule  for  composite  formation  is  largely  inferred  from  such 
information  as  the  attribute  values  of  its  constituents.   Inference  rules 
using  Baysean  strategies  and  subjective  probabilities  of  Ward  Edwards 
in  fact  all  the  armamentarium  of  classical  pattern  recognition  theory  will 
be  developed  for  this  latter  purpose. 

2.5  Experimental  Tests  of  the  Model 

To  test  the  feasibility  of  the  global  model  for  picture  processing, 
Ian  to  show  a  practical  example  for  a  small  class  of  pictures  in  at 
least  one  application  area. 

At  present  the  proposed  application  area  is  scanning  electron 
:ope  pictures,  such  as  those  of  polycrystalline  materials,  which  display 
gree  of  structural  organization.   These  pictures  exceed  the  complexity 
line  patterns.  At  the  same  time,  they  are  not  familiar  scenes  in 
ry  difficult  to  separate  the  part  of  the  recognition  task  which 
rent  visual  structure  from  the  part  which  has  been  learned 
. 
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2.5.1  Picture  Input 

The  initial  input  is  an  encoding  of  the  original  picture  into 
basic  sets.   These  can  include  lines,  textured  areas,  and  regions  with 
attributes.   Sufficient  preprocessing  is  assumed,  so  that  the  next  higher 
logical  organization  of  the  basic  sets  are  cells. 

2.5-2  Mosaic  Determination 

The  input  data  is  partitioned  into  the  cells  of  the  initial  mosaic. 
From  the  edge  and  neighborhood  information  it  will  in  general  be  necessary 
to  imply  the  complete  boundaries  of  the  mosaic  cells.   Here  the  factors  that 
make  up  a  good  mosaic  must  be  determined. 

2.5.3  Relation  Determination 

The  relationships  among  subparts  of  the  picture  which  are  important 
to  composite  formation  must  then  be  determined.   The  structural  composition 
of  the  picture  should  be  expressed  in  terms  of  these  relationships. 

2.5«^  Composite  Formation 

The  capability  of  fitting  partitioned  objects  to  named  prototypes 
is  necessary. 

To  this  end  composite  formation  will  be  attempted  by  applying 
graph  transformations  to  relation-linked  cells.   Sufficient  information  must 
be  maintained  from  the  lower  level  relational  parsing  to  match  with  a  defined 
prototype.   The  attributes,  relations  and  structures  must  also  be  considered 
with  respect  to  the  efficiency  of  alternate  matching  techniques. 

It  may  be  that  grammar  directed  parsing  rules  can  be  defined 
equivalently  in  terms  of  name- independent  graph  transformations  so  that 
both  techniques  are  interchangeable.   Efficiency  considerations  will  influence 
the  interactions  between  these  levels  of  parsing.   At  the  levels  where 
equivalences  exist  between  grammar- directed  techniques  and  relational  techniques, 
factors  to  select  preferred  methods  should  be  identified. 

2.5-5  Specification  Matching 

To  check  the  validity  of  the  processing,  the  composite  object 
identified  must  be  matched  to  a  specification  which  formed  part  of  the  input 
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scanning  instructions  and  the  associated  training  set.   The  fact  that  a  composit* 
was  formed  does  not  guarantee  that  it  will  be  constituent  of  an  optimal  or 
even  a  successful  parse  (scene  description).   There  will  be  ambiguous  choices 
for  objects  at  this  level  which  must  be  further  resolved.   Decision  factors 
for  optimal  choices  must  be  determined. 

2.5.6  Performance  Measurement 

To  compare  and  evaluate  the  decision  strategies  which  guide  the 
parse,  measures  of  efficiency  must  be  recorded.   The  time,  as  well  as  other 
measures  of  the  use  of  attributes,  relations,  and  rules  will  be  recorded 
for  each  strategy.   We  will  attempt  to  identify  the  relevance  of  subparts 
of  the  proposed  model  and  compare  the  capabilities  of  each  feature.   For 
example,  the  use  of  an  available  attribute,  such  as  color,  may  greatly 
increase  the  efficiency  of  the  process. 

2.6  Related  Work 

2.6.1  Linguistic  Models 

Many  picture  processing  systems  whose  chief  component  is  a  language 
for  defining  structures  have  been  reported  in  the  literature.   Most  of  these 
systems  deal  with  lines  as  terminal  elements  and  use  specialized  cases  of  the 

general  set-replacement  rule  mentioned  under  the  structure  space  description. 

11        1*3  "VT 

systems  of  Anderson  ,  Evans   ,  Feder   ,  and  Shaw  '  conform  to  this 

description. 

These  systems  fit  the  general  two-level  model  of  Miller  and  Shaw29, 
are  described  by  a  low-level  primitive  or  terminal  element  description, 
lented  by  a  hierarchic  description  which  describes  how  primitives  are 
to  higher  level  structures.   Semantic  components,  such  as  variables, 
iated  with  a  primitive  or  an  hierarchic  description.   The  hierarchic 
1  defined  by  a  generative  grammar  which  is  then  used  to  parse  a 
picture. 

mi tat ions  of  present  systems  are  inefficiency  and  a  line 
.on.   The  general  inefficiency  is  due  to  the  fact  that  the 
es  are  implemented  by  top-down  exhaustive  set-partitioning 
nherently  slow.   Many  candidate  replacement  sets  exist 
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for  each  rule  and  each  of  them  must  be  checked  for  the  proper  relationships 
and  attributes  before  attempting  a  replacement. 

Name-independent  techniques  attempt  to  do  partitioning  in  a  more 
efficient  manner.  Properties  of  relations  basic  to  a  class  of  objects  may- 
be needed  to  improve  the  partitioning  for  that  class. 

2.6.2  Data  Base  Models 

In  the  "world  model"  for  scene  description  being  developed  at 

31 
Stanford  ,  prototypes  and  instances  of  objects  are  defined  in  terms  of  attri- 
bute, object,  value  triplets.   Here  prototypes  correspond  to  elements  of  our 
structure  space  and  instances  to  objects  in  the  data  base. 

Attribute-value  pairs  seem  to  be  a  convenient  terminal  representation 

20 
for  data  elements.   Hsiao  and  Harary   use  this  as  a  basic  notation  to  define 

a  formal  system  for  information  retrieval  from  files. 

2.6.3  Heuristic  Systems 

The  program  of  Guzman      demonstrated  the  feasibility  of  what  we 
have  called  name- independent  parsing  of  objects.   Heuristics  involving  object 
vertices  were  used  to  merge  objects.   We  have  been  able  to  show  that  Guzman's 
work  is  conveniently  expressible  in  our  proposed  model.   In  this  case,  planar 
faces  form  the  basic  cells,  and  vertex  types  imply  relationships  between 
cells.   The  composite  formation  process  of  cell  merging  can  be  implemented 
by  simple  graph  transformations  with  two  classes  of  relations. 

2.6.U     Non-Linguistic  Models 

Fennema   has  formulated  "a  region-oriented  structure"  to  represent 
scenes.   He  proposes  merging  equivalent  connected  points  to  elementary  regions 
and  then  further  merging  under  additional  criteria.   These  criteria  should  be 
expressible  by  relations  between  cells. 
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3.   PLANE  PARALLEL  PROCESSING  STRATEGIES 

An  image  processing  strategy  to  be  effective  must  also  be  compu- 
tationally efficient  -  a  criterion  which  can  only  be  evaluated  in  the  light 
of  given  image  processing  hardware.   It  is  our  intention  here  to  devise, 
implement  and  evaluate  parallel  processing  strategies  appropriate  to  the 
Pattern  Articulation  Unit  of  the  Illiac  III  Computer. 

Specifically  we  recall  that  a  picture  with  an  extended  gray  scale 
range  is  commonly  encoded  over  a  regular  raster  of  unit  cells.  This  array 
of  digitized  scalar  light  intensity  values  can  be  conceptually  viewed  in 
two  complementary,  if  equivalent,  ways.   In  the  first,  or  conventional 
cellular  representation,  the  encoded  picture  is  viewed  as  a  matrix  of  digitized 
values  (typically  of  uniform  1,  2,  k   or  8  bits  per  cell).   Alternatively  we 
can  view  the  image  topographically  as  planar  sets  of  all  cellular  positions 
having  a  gray  level  amplitude  >_  the  assigned  (planar)  threshold  value.   (SeeFif 
ures  7-8).   The  planar  representation,  which  has  recently  enjoyed  new  popu- 
larity, forms  the  basis  of  the  PAX  picture  processing  language  and  the  Pattern 
Articulation  Unit  of  Illiac  III.   (PAX  was  originally  introduced  as  a  simu- 
lation language  for  the  PAU. ) 

Conceptually  the  planar  representation  emphasizes  planar  sets  of 
cells  having  a  common  gray  level.   As  the  mesh  size  approaches  zero,  the 
asymptotic  properites  of  the  representation  are  treated  in  the  measure  theory 
associated  with  the  Lebesque  integral  (in  2-dimensions ) .   The  conventional 
matrix  representation  finds  its  counterpart  in  the  Riemann  integral  with  its 
cell-by-cell  identification  of  gray  level.   Preprocessing  filters  in  the 
planar  representation  are  typically  homogeneous  boolean  transforms  of  the 
surround  (both  neighboring  cells  and  neighboring  planes)  of  each  picture 
point.   For  scalar  arrays,  linear  transforms  with/without  thresholding  are 
more  commonly  used. 

With  this  preamble  we  observe  that  i)  little  formal  theory  for 
design  of  preprocessing  (image  enhancement /edge  detection)  filters 
using  the  planar  representation  has  been  developed  to  date,  and  ii)  there 
sast  three  classes  of  images  for  which  these  procedures  are  known 
tive — and  warrant  formal  development.   We  propose  to  rectify  both 
. 
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Figure  7  Planar  Organization  of  the  Pattern  Articulation  Unit 
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Areas  of  advantageous  application  for  plane  parallel  processing 
includes  the  following: 

1)  Shape  determination  from  scenes  of  shaded  objects.   Shading 
implies  the  orientation  of  each  differential  surface  element. 
Different  physical  laws,  however,  apply  to  the  reflection  of 
light  as  opposed  to  (for  SEM  pictures)  the  emission  of 
secondary  electrons.   Given  an  object  with  gradual  variation 
of  shading,  the  task  is  to  find  a  suitable  cellular  decomposi- 
tion, and  to  infer  from  this  mesh  a  computer-graphics  type 
representation  of  the  spatial  structure  of  the  shaded  surface. 

2)  Fast  articulation  of  (quasi)  line-like  images.   Fast  tracking 
algorithms  will  be  sought  for  patterns  characterized  by 
simple  lines  with /without  noise.   Suitable  classes  of  pictures 
includes  engineering  drawings,  nuclear  tracks,  chromosomes 
and  Golgi-stained  brain  tissue. 

3)  Detection  of  differential  changes  in  scene  representation  with 
motion.   The  important  application  of  image  processing  to  cine- 
photography  and  television  is  obviously  predicated  upon  the 
development  of  strategies  which  detect  incremental  change  in 
the  cellular  mosaic  representation  of  the  scene  over  time. 
Techniques  are  of  interest  here  which  avoid  ab  initio  analysis 
of  each  frame  and  therefore  exploit  per served  or  translated 
features. 

It  is  the  intention  of  this  proposal  to  investigate  these  plane 
parallel  strategies  for  image  processing,  to  seek  where  possible  formal 
methods  of  more  general  applicability  and  to  quest  for  their  earliest 
implementation  within  the  Pattern  Articulation  Unit  of  the  Illiac  III. 
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k.      LIST  PROCESSING  IN  TAXICRINIC  PROCESSORS 

k.l     Function  of  the  Taxicrinic  Processors 

The  Taxicrinic  Processors  are  the  central  control  units  of 
Illiac  III.   The  Taxicrinic  Processors  are  primarily  responsible  for  the 
execution  of  user  programs,  that  is,  to  oversee  the  operations  of  the 
Pattern  Articulation  Unit,  the  Arithmetic  Unit  and  to  initiate  input/ 
output  operations  in  the  I/O  Processors  by  making  requests  to  the  Interrupt 
Unit. 

The  principal  activity  of  a  Taxicrinic  Processor  is  the  manipula- 
tion, search  and  systemization  of  abstract  graphs  (bilateral  list  structures) 
which  have  been  produced  from  the  pictorial  input  to  the  Pattern  Articulation 
Unit.   The  name  "Taxicrinic"  comes  from  two  Greek  words:   tagia  meaning 
"arrangement"  or  "pattern",  and  Kpivo)  meaning  to  "judge",  thus  indicating 
the  TP's  general  purpose,  which  is  to  syntactically  analyze  digitized 
pictures  and  other  material  which  can  be  cast  into  the  form  of  a  directed, 
labeled  graph. 

U.2  Design  Features 

The  Taxicrinic  Processors  can  address  operands  in  either  a  paged 
or  unpaged  (contiguous  storage)  environment.   Automatic  reference  to  the 
Segment  Name  Table  insures  that  operands  are  always  referenced  in  a  completely 
relative  (relocatable)  manner.   Pointer  stacks  are  provided  to  expedite 
addressing  of  trees  (i.e.  "structures"  in  PL/l  notation)  and  to  permit 
automatic  nested  macro  calls  (see  below). 

Central  to  the  design  is  the  capability  of  the  TP  to  execute  a 
of  procedure  calls  (and  returns)  entirely  in  hardware  —  including  the 
mapping  of  actual  onto  formal  parameters  (operands).   This  facility  means 
extensive  use  can  be  made  of  read-only  memory  —  of  particular 
tage  in  a  time-sharing  system,  and  ii)  an  interpretative  translation 
>cedure  can  be  used  efficiently  to  implement,  for  example,  an  extensive 
lis-      ssing  repertoire. 
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List  processing  has  also  been  designed  into  the  processor  by 
including  instructions  for  the  control  of  Available  Space,  and  for  the 
insertion/deletion  of  a  list  cell. 

In  addition  the  TP  must  interpret  Pattern  Articulation  Unit  instruc- 
tions, much  as  a  more  conventional  CPU  must  communicate  with  an  Arithmetic 
Unit.   Where  possible  operand  transmittal  has  been  based  -upon  buffering  in  the 
Operand  Stack  of  the  Taxicrinic  Processor,  an  array  of  32  bytes  of  fast 
storage  with  automatic  refill/unload  facilities  coupling  to  that  portion  of 
the  stack  in  core. 

U.3  Proposed  Investigation 

The  Taxicrinic  Processor  design  was  in  part  the  outgrowth  of 
examining  a  number  of  list  processing  languages  and  searching  for  that  structure 
which  seemingly  offered  notable  advances  over  current  machine  organizations. 
Besides  the  explicit  list  processing  instructions,  of  central  importance  here 
is  the  built-in  macro  facility  whereby  macro  calls  (and  returns)  are  given  hard- 
ware implementation.   In  the  interim  however  the  TP  design  has  developed  a  life 
of  its  own  as  one  sought  an  internally  consistent  hardware  structure. 
During  this  same  period  available  list  processing  languages  have  also  evolved, 
although  disappointingly  few  new  developments  such  as  the  introduction  of 
ring  structures  have  emerged. 

It  is  now  proposed  that  concurrent  with  final  checkout  of  the  TP, 
the  appropriateness  of  the  TP  as  a  vehicle  for  the  implementation  of  list 
processing  languages  be  reevaluated.   We  reiterate:   our  immediate  interest 
here  is  not  in  the  design  of  yet  another  list  processing  language  (e.g. 
LISP,  SLIP,  SNOBOL  k   for  lists  and  GRAIL  for  ring  structures).   Nor  is  our 
immediate  interest,  at  this  time,  in  the  full  implementation  of  these  languages. 
Rather  we  wish  to  investigate  the  match  between  the  machine  (i.e.  TP)  and 
this  class  of  language:   can  short  efficient  implementation  be  given  to 
specifically  list  processing  portions  of  these  languages.   In  other  words  we 
are  driven  by  the  following  questions: 
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1)  Are  the  hardware  instructions  of  the  TP  logically  consistent? 
Do  they  steer  their  way  gracefully  through  the  seemingly 
endless  interrupts  of  page  out,  available  space  empty,  operand 
stack  overflow,  etc.? 

2)  Does  the  TP,  appropriately  scaled  for  circuit  speed,  fulfill 
its  promise  of  providing  a  significant  speedup  in  the  execution 
of  the  core  macros  of  contemporary  list  processing,  and 

3)  Can  a  simulation  model  be  developed  that  monitors  the  internal 
machine  data  transfers  in  a  more  global  way  and  allows  one  to 
look  for  possibly  more  efficient  machine  organizations  for 
list  processing? 
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5.   APPLICATIONS 

5. 1  Task  Environment 

Several  considerations  enter  into  the  choice  of  an  appropriate 
domain  for  the  imagery  of  the  scanning  robot:   l)  The  projected  application 
warrants  the  development  of  a  practical  scanning  tool;  (2)  The  characterization 
of  the  image  as  a  limited  number  of  regions  and  edges  (i.e.  cellular  mosaic) 
must  be  natural  to  the  image  domain;  and  (3)  The  objects  to  be  described 
are  adequately  complex,  and  yet  are  not  overburdened  with  non-visual  organi- 
zation.  Specifically  scenes  from  daily  life  impose  additional  difficulties 
in  this  later  regard. 

For  these  reasons,  and  because  of  its  potential  significance  for 
large  scale  integration,  we  have  chosen  to  emphasize  Scanning  Electron 
Microscope  pictures.   Both  in  its  applications  to  materials  science  and  to 
histology  and  subcellular  biology,  the  SEM  offers  an  exceptional  tool  to 

explore  a  world  only  recently  revealed.   In  addition  recent  advances  give 

12  3 
promise  of  direct  readout  of  molecular  and  pauciatomic  structures. 

And  if  robotics  seems  inappropriate  to  this  macromolecular  domain, 
let  me  quote  Fernandez-Moran  :   "  -  in  a  deeper  sense  we  realize  that  electron 
microscopy  represents  the  ultimate  direct  extension  of  eye  and  hand.   Man 
may  hereby  be  granted  strange  new  freedom  and  capabilities  of  "seeing"  and 
"doing"  in  those  pivotal  microcosmic  domains  of  life  which  are  still  beyond 
mind's  grasp." 

Further,  the  Scanner-Monitor-Video  of  Illiac  III  Computer  System 
could  be  readily  expanded  to  incorporate  an  SEM  as  yet  another  "scanner".   It 
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is  proposed  here  however  to  concentrate  on  image  description  strategies 
which  could  he  evaluated  from  film  images.   The  loss  of  flexibility  and  the 
ability  to  readily  interdigitate  with  the  three  common  modes  of  SEM  operation: 
emissive,  conductive  and  luminescent,  can  be  compensated  by  a  later  direct 
access  to  an  SEM,  should  the  scanning  strategy  discussed  below  prove  appli- 
cable. 

As  implicated  above  the  SEM  provides  an  extraordinarily  versatile 
tool  for  the  examination  of  large  scale  integrated  circuitry.  Not  only  can 
one  monitor  the  detailed  geometry  of  the  diffusion  and  deposition  processes 
but  under  suitable  operating  conditions  the  local  circuit  fields  can  also 
be  monitored.   Checkout  of  developmental  LSI  circuitry  of  the  future  will 
likely  entail  the  use  of  an  electron  beam  probe  (i.e.  the  scanning  electron 
microscope)  quite  as  today  the  technician  uses  a  manually  positioned  contact 
probe.   Visual  models  of  the  circuit  terrain  can  be  derived  from  the  descrip- 
tion of  the  multiple  masks  controlling  the  etching  process.   The  visual 
models  here  could,  for  example,  be  expressed  in  the  computer  graphics  package 
that  has  been  developed  here  under  the  AEC  computer  graphics  program.   Image 
recognition  as  applied  to  SEM  pictures  or  integrated  circuitry  then  complements 
these  computer  graphic  strategies,  and  provides  a  mechanism  for  this  laboratory 
to  remain  abreast  of  the  developments  in  large  scale  integration:   a  field 
critical  importance  to  future  trends  in  computer  architecture. 

5. 2  Two-Dimensional  Scene  Analysis 

Before  moving  immediately  to  the  full  complexities  of  the  3-dimensiona 
world  of  the  SEM,  it  was  felt  that  a  better  initial  strategy  would  be  to 
practice  the  problems  of  cellular  decomposition  on  materials  ideally  suited 

purpose:   typical  microstructures  encountered  in  optical  metallography. 
:ordingly  a  graded  sequence  of  2-dimensional  metallographic  sections  of 
icreasing  difficulty  have  been  chosen  (see  Figure  9)*.    For  picture  A 

pie  grain  boundary  contrast)  typical  parameters  sought  include  mean  linear 
•cept,  grain  section  size  distribution,  anisotropy  and  aspect  ratio, 
simple  particle  distribution)  parameters  desired  include 


rographs  were  selected  for  us  by  Mr.  John  B.  Woodhouse  of 
MRL  scanning  electron  microscope  facility. 
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A.   Simple  grain -"boundary- 
contrast 


B.   Simple  particle  distribution 


Particles  in  matrix  with 
grain  boundaries 


D.  Two  phase  sample  with  one 
phase  having  a  gradation 
of  tones 


contrast  only 
(olor  discrimination  suppressed) 


F.   .Multiphase  sample 

(color  discrimination  suppressed) 
Figure  9.   Typical  Microstructures  Encountered  in  Optical  Metallography 


particle  mean  area  in  section,  size  distribution,  anisotropy  and  aspect 
ratio.   These  first  two  images  are  currently  interpretable  "by  the  quant imet 
analog  image  processor,  and  hence  present  a  realistic  lower  bound  upon  the 
demanded  capabilities  of  the  proposed  image  processing  system. 

A  second  class  of  2-dimensional  scenes  are  presented  by  large 
scale  integrated  circuits,  as  seen  under  the  light  microscope.  Again  the 
limited  depth  of  focus  of  the  light  microscope  projects  the  intrinsically 
3-dimensional  circuit  scene  onto  the  image  plane.   Directly  applicable  here 
will  be  algorithms  for  the  fast  articulation  of  line-like  images  (see  Section 
3,  topic  2) . 

Finally  to  explore  the  possibilities  for  detecting  differential 
changes  in  scene  representation  with  motion  (see  Section  3,  topic  3),  we 
propose  to  explore  time-lapse  cinemicrophotography .   Specifically  we  have  in 
our  possession  elegant  films  of  the  locomotion  of  cancer  cells  in  vivo 
compared  with  normal  cells  in  the  rabbit  ear  chamber.   Here  of  course  the 
"cells"  of  the  mosaic  scene  decomposition  are  largely  biological  cells: 
lymphocytes ,  granulocytes ,  macrophage ,  monocytes ,  fibroblasts  and  cancer 
cells. 


5.3  Three-Dimensional  Scene  Analysis 

Typical  SEM  micrographs  are  exhibited  in  Figures  10-12.   In  the 
scene  dendrites  of  a  ferritic  material  have  been  vapor  deposited  some 

distance  from  the  melt.   Note  how  the  four  faces  on  each  dendrite  (the  pyramids) 

2 
parallel.   Magnification  here  is  2000X.    Both  in  this  image  and  in  the 

3 
econd  micrograph  of  mangonese  oxide  crystals  the  analogy  with  the  idealized 

blocks,  as  treated  by  Guzman,  is  readily  apparent.   And  intentionally 

lere  we  can  reasonably  anticipate  some  measure  of  success  with  the 

ocessing  model. 

These  latter  micrographs  also  provide  exemplar  images  to  practice 

ing  plane  parallel  processing  algorithms  of  Section  3:   shape 


'essor  Sumner  Wood,  Jr.  ,  Department  of  Pathology,  John 
y. 

.  Harmer.   Micrographs  taken  by  J.  Temple  Black, 
oscopy,  University  of  Illinois,  Urbana, 
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10.   The  Melt  Interface  of  a  Silica  Brick  From  an  Open  Hearth  Furnace  (see  text) 
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1,  side  Crystals  with  Metallic  Silver  Precipatate 
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determination  from  scenes  of  shaded  objects.   This  situation  is  very 
characteristic  of  the  repertoire  of  less  complex  SEM  pictures. 

Finally  Figure  12,   Radiolaria  900X  by  Phil  Sandberg, 
is  introduced  to  show  the  extensive  depth  of  field  in  the  SEM.   Given 
stereo  pairs  of  SEM  micrographs,  it  is  possible  to  use  photogrammetric 
techniques  to  compute  absolute  dimensions  of  all  member  parts.   We  shall, 
however,  leave  this  task  for  a  later  time. 


Courtesy  of  J.  Temple  Black,  Center  for  Electron  Microscopy,  University 
of  Illinois,  Urbana. 
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Figb-        adiolaria  900X  by  Phil  Sandberg 
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6.  FACILITIES  AVAILABLE 

6.1  Scanner-Monitor-Video  System 

The  Scanner-Monitor- Video  (SMV)  system  of  Illiac  III  provides  a 
general  purpose,  high  resolution  scan/display  facility.   Rasters  of  all  formats 
prescribed  in  the  SJCC  article  ("Parametric  Description  of  a  Scan-Display 
System",  by  L.  A.  Dunn,  L.  N.  Goyal ,  B.  H.  McCormick  and  V.  G.  Tareski ,  Spring 
Joint  Computer  Conference  1969;  Appendix  2)  are  now  operational.   In  addition, 
the  intermachine  link  between  the  core  memories  of  Illiac  III  and  the  PDP-8 
computer  of  Professor  C.  W.  Gear's  graphics  group  is  also  operational. 

For  prototype  studies  with  the  SMV  scan/display  system  a  general 
purpose  program  (SEP)  has  been  written  .   This  program,  designed 
by  R.  T.  Borovec,  includes  routines  to  edit  scan  parameters,  run  the  scanner, 
read  and  write  digitized  pictures  on  DECTAPE  (for  subsequent  processing  on 
either  the  PDP-8  or  the  360/75  running  PAX),  print  gray  scale  pictures  and 
check  the  integrity  of  the  various  hardware  links.   All  commands  are  issued 
through  a  TTY  terminal. 

We  have  been  very  pleased  with  the  linearity  and  reproducibility  of 
the  rasters,  the  quality  of  the  gray  scale  encoding  (currently  limited  to  l6 
levels)  and  the  high  data  ratio  (5  MHz  bit  rate  output).   The  2-1/2  years  of 
intensive  analog  circuit  design  work  (largely  by  Divilbiss  and  Franco)  is 
now  proving  to  have  been  a  worthwhile  investment.   Comprehensive  hardware 
documentation  is  also  being  prepared.   The  extensive  control  of  the  SMV  system 
uses  7^-00  Series  dual  inline  packages,  and  could  be  readily  copied  at  other 
AEC  installations. 

Currently  we  are  evaluating  the  SMV  system  by  looking  at  applications 
to  polycrystalline  materials  research,  exfoliative  cytology  and  the  scanning 
of  brain  nuclei. 

6-2   Illiac  III  Computer  System 

Illiac  III  is  an  experimental  computer  being  designed  and  constructed 
by  the  Department  of  Computer  Science  as  a  first  instrument  to  explore  the 
potentialities  of  high  speed  image  processing.   Concurrent  with  the  computer 
design  work,  the  AEC  contract  AT(ll-l)-10l8  has  supported  research  into  the 
theory  of  image  processing. 


"Richard  T.  Borovec,  The  Illiac  III  Scanner  Exerciser  Program  (SEP),  DCS 
File  No.  830,  April  1970. 
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The  I Iliac  III  Computer  System  can  be  described  in  terms  of  Ik 
constituent  subsystems  of  processors,  units  and  peripheral  device  groups  as 
follows  (see  Appendix  1,  Figure  l): 

i)  Central  System  (6  subsystems) 

ii)  I/O  System  (3  subsystems) 

iii)  Peripheral  System  (k   subsystems )* 

iv)  Power  Distribution  System  (global  to  all  subsystems). 

Only  two,  the  Fast  Core  Storage  Units  and  the  Low  Speed  Terminal  Network, 
are  of  commerical  design.   The  other  constituent  subsystems  have  been  designed 
almost  entirely  as  part  of  the  graduate  research  program  of  the  Department  of 
Computer  Science. 

This  massive  design /development  program  was  not  undertaken  lightly. 
Rather,  central  to  the  contract  program  has  been  the  study  of  the  computer 
architecture  appropriate  for  high  speed  image  processing.   These  peripheral 
devices,  units  and  processors  must  be  able  to  talk  to  one  another — and  with 
minimum  data  format  conversion:   the  Scanners  feed  the  Pattern  Articulation 
Unit,  whose  output  in  turn  is  interpreted  by  a  Taxicrinic  Unit.   This  global 
integration  of  the  design  has  been  costly  but  integral  to  the  integrity  of 
the  system  design. 

6. 3  Scanning  Electron  Microscope  Facility 

The  AEC-supported  Materials  Research  Laboratory  of  the  University 
of  Illinois,  Urbana,  is  setting  up  a  new  facility  for  electron  microscopy. 
A  JSM-3  Scanning  Electron  Microscope,  Japan  Electron  Optics  Laboratory  Co., 
has  been  purchased  and  is  now  being  installed  in  the  laboratory. 

This  facility  will  be  under  the  direction  of  Mr.  John  B.  Woodhouse, 
who  has  formally  offered  us  full  cooperation.   Mr.  Woodhouse  has  prior 
industrial  experience  in  the  design  of  analog  instruments  for  metallurgical 
and  cellular  image  processing.   He  therefore  sees  the  proposed  computer- 
based  image  processing  system  as  a  natural  next  step.   Our  interface  with 

SEM  facility  will  be  Dr.  Jerry  Chih-li  Chen,  a  solid  state  physicist  now 
returning  for  a  Ph.D.  degree  in  Computer  Science. 

The  proposed  collaboration  has  been  cleared  with  Professor  Robert 
r,  director  of  the  Materials  Research  Laboratory,  and  has  his  full 
su; 


r-Monitor-Video  System  of  Section  8.1.  (Appendix  l). 
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7.   PRELIMINARY  INVESTIGATIONS 

7.1  Properties  of  Relations 

As  a  result  of  the  composite  formation  ideas  we  attempted  to  do 
some  theoretical  work  to  determine  general  rules  for  composite  analysis  given 

a  directed  graph. 

,   28 
First,  in  DCS  File  No.  762   ,  we  studied  "basic  set  related  properties 

of  binary  relations.   These  are  properties  such  as  reflexive,  symmetric, 
transitive,  and  cyclic,  which  do  not  depend  on  additional  structure  on  the 
set.   We  defined  nine  independent  properties  and  showed  that  exactly  forty 
consistent  classes  of  property  values  existed.   This  characterization  appears 
useful  for  any  general  relational  or  graph-oriented  system  for  the  purpose  of 
reducing  the  complexity  of  graphs  by  eliminating  redundant  edges. 

Usually  there  is  some  structure  in  the  set  of  objects  under  con- 
sideration.  For  example,  if  a  composite  element  is  formed  by  the  set  union  of 
subelements,  then  we  can  consider  the  power  set  of  the  set  of  basic  objects  and 
the  structure  is  that  of  a  Boolean  algebra.   In  this  case  we  would  like  to 
know  how  relations  can  be  extended  from  constituents  to  composites,  and  classify 

relations  according  to  types  of  composite  formation  properties. 

35 

Thus,  in  another  publication   ,  we  considered  systems  of  binary 

relations  between  latices  where  the  meet  and  join  operations  established  the 
basic  structure.   We  defined  thirteen  properties  of  binary  relations  which 
depend  on  the  lattice  structures. 

Further  theoretical  work  in  this  area  should  involve  a  consideration 
of  special  types  of  lattices,  such  as  modular  and  geometric.   Each  type  of 
lattice  will  imply  certain  of  the  composite  formation  and  relation  extension 
properties.   The  goal  here  is  to  find  the  mathematical  structure  best  suited 
to  picture  processing  operations. 

7.2  Graph  Transformations 

In  order  to  implement  name- independent  procedures  we  would  like  to 
imply  graph  transformations  from  the  properties  of  individual  relations.   In 
DCS  Report  No.  368   ,  we  considered  a  set  of  simple  graph  transformations  which 
seemed  plausible  according  to  some  previous  experimentation,  and  determined 
which  transformations  were  implied  by  the  previously  defined  properties. 
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One  useful  concept  for  selecting  preferred  transformations  is 
the  idea  of  an  information  lossless  transformation.   Transformations  which 
can  "be  shown  to  "be  reversible,  i.e.  the  transformation  and  its  inverse  are 
always  valid,  can  be  performed  with  no  loss  of  information. 

7.3  Structural  Component 

Our  early  formulation  of  structural  description  and  structural 
operations  (graph  transformation)  was  expressed  in  the  WEB  language.    The 
language  arose  from  a  need  to  describe  separately,  readily,  and  concisely,  the 
essential  logical,  geometrical,  and  electrical  characteristics  of  a  contemporary 
digital  processor  in  a  form  amenable  to  symbolic  manipulation.   The  WEB 
logical  block  allowed  the  specification  of  structural  levels  in  a  manner 
which  is  very  similiar  to  the  NAPE  (n-attaching  point  entity)  used  in  the 
recent  picture  processing  system  of  Feder  ^    .   The  WEB  blocks  are  now  generalized 
to  the  set- replacement  rules  of  the  picture  specification  language. 

7.3.1  Language  for  Picture  Specification 

A  number  of  graphical  or  pictorial  descirption  languages  have 
been  studied  to  determine  desirable  features  of  an  image  processing  language, 
and  to  evaluate  for  each  language  its  satisfaction  of  stated  a  priori  criteria. 
After  a  brief  survey  of  some  of  the  most  representative  of  these  languages,  a 
new  image  processing  language  IC0N  was  defined. 

The  ICON  language  can  be  reinterpreted  as  specifying  set-replacement 
rule  grammars.   A  rule,  P  ,  replaces  a  set  of  n  £_ 1  elements  by  a  set  of  one 
element.   The  rule  may  be  represented  by  the  diagram  of  Figure  13-   Here  the 
set  {Xpl,  Xp2,  ...,  Xpn  }  is  replaced  by  the  set  {Xpo}.   Each  element,  Xpj , 
has  an  associated  set  of  attributes,  A(Xpj).   A  set  of  attribute  mapping 
functions,  {fpja}  determines  the  values  of  each  attribute  a  in  A(Xpj)  from 
the  values  of  attributes  of  the  other  elements.   Sets  of  relations  between 
the  attributes  of  each  element,  {R(A(Xpj ) ) } ,  and  between  all  elements, 
{R(Xpl,  Xp2,  .. .  .  ,  Xpn  )},  can  also  be  specified. 

The  set-replacement  rule  and  attribute  assignments  can  be  made  if 
found  which  satisfy  all  of  the  relations. 


.al. ,  "The  WEB  System,  Part  II:   A  Formal  Description  of  the  WEB 
•  ",  DCS  Report  No.  232,  University  of  Illinois,  June  19,  1967  • 
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STRUCTURAL    RULE; 


XnO 


ATTRIBUTE     MAPPINGS: 

A(Xpo) 

•    •    • 

A 


>  f 

V 


•      •      •  •      •     • 


Xpi       Xp2      ...        Xpnp  A(Xpl)    A(Xp2)-   •   •    A(Xpnp) 


CONSTRAINTS 


{r(a(xp1))}      {r(a(xp2))} 


•     •     • 


{R(A(XPnp))} 


{R(Xpi,Xp2,  •  •  -.Xpn,,)} 


Figure  13.      Set-Replacement  Rule 
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7. h     Plane  Parallel  Processing 
7.1+.1  Picture  Description 

1» 
Much  work,  summarized  and  referenced  in  Narasimhan   ,  has  "been 

done  on  a  descriptive  scheme  for  classes  of  pictures  based  on  labelling 

techniques  using  parallel  processing  algorithms.  The  structures  of  syntactic 

descriptive  models  was  studied  by  considering  their  specific  application  to 

2' 
bubble  chamber  pictures.  A  program  called  BUBBLE  TALK  ,  was  designed  for 

online  conversation  in  the  context  of  bubble  chamber  pictures. 

1 .k.2     Scanning  Procedures 

3* 

The  Bubble  Scan  I  program  by  Narasimhan   was  concerned  exclusively 

with  the  scanning  of  one  view  of  a  bubble  chamber  stereotriad.   Input-output 
features,  data  structures,  and  scanning  procedures  were  implemented  on  the 
IBM-7091+  computer. 

7-1*. 3  PAX  II 

In  the  course  of  design  studies  for  the  Pattern  Articulation  Unit 
(PAU),  it  proved  useful  to  introduce  a  programming  language  PAX,  initially 
considered  as  a  simulator  for  the  PAU.   This  language  introduced,  we  believe, 
the  first  systematic  use  of  connectivity  information  in  image  analysis.   The 
PAX  concepts  were  systematized  and  extended  in  a  series  of  basic  papers  by 
Narasimhan   (than  at  this  laboratory).   The  PAX  language,  as  PAX  II,  has  recenl 
been  implemented  for  the  IBM  360/75  by  J.  W.  Snively,  Jr.  and  E.B.  Butt, 
formerly  of  the  University  of  Maryland,  and  is  used,  for  example,  by  Kirsch  anc 
Lipkin  in  their  experimental  studies  of  bio-medioal  image  processing  at  the 
National  Institute  of  Health.   In  addition  PAX  has  been  taught  as  the  language 
for  picture  preprocessing  and  feature  extraction  in  a  recent  commercial  course 
on  Image  Processing  by  Cybex  Associates,  Inc. 

In  an  effort  to  provide  a  bridge  between  the  present  IBM  360 
tlation  and  the  PAU  hardware  execution  of  local  feature  extraction  algoritl 
the  Snively-Butt  version  of  PAX  II  was  adapted  to  the  Illinois  installation. 
Rej        JlU,  "The  PAX  II  Picture  Processing  System  at  the  University  of 
Illi.'    ,  March  1969,  edited  by  R.  T.  Borovec  describes  the  translator. 


i* 


' .  R.  Narasimhan,  "Syntax-Directed  Interpretation  of  Classes  of  Pictures", 
CACM,  Vol.  9,  No.  3,  March  1966,  166-173. 

1 .  R.  Narasimhan,  J.  R.  Witsken,  and  H.  Johnson,  "Bubble-Talk:  The  Structure 
of  a  Program  for  On-Line  Conversation  with  Illiac  III,  DCS  File  No.  60U, 
University  of  Illinois,  July  2,  196k. 

'.  R.  Narasimhan,  "BUBBLE  SCAN",  DCS  Report  No.  167 ,  University  of  Illinois, 
August  19,  196 U. 
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8.   SUMMARY 

Strategy  is  proposed  to  permit  personnel  (not  trained  in  computer 
technology)  to  input  scanning  instructions  directly  to  the  image  processing 
system. 

To  this  end  a  formal  model  for  picture  processing  is  proposed. 
Constituents  of  the  model  include  local  preprocessing,  partitioning  of  the 
scene  into  a  cellular  mosaic,  clustering  of  picture  sub-elements  (initially 
the  cells,  or  regions,  of  the  cellular  mosaic)  into  composites,  and  the 
matching  of  the  hierarchial  structure  obtained  to  prototype  objects. 

Particular  emphasis  is  placed  in  the  proposal  upon  developing 
effective  and  efficient  procedures  for  image  analysis.   Attention  is  given 
to  signal  detection  theory  (for  edge  detection/texture  discrimination), 
plane  parallel  picture  processing  (for  pattern  articulation),  and  efficient 
machine  implementation  of  list  processing( to  transform  the  graph  representa- 
tion of  scenes ) . 

Of  the  diverse  applications  discussed  in  the  proposal,  automatic 
photo  interpretation  of  Scanning  Electron  Micrographs  has  been  singled  out 
for  intensive  investigation.   The  microstructure  of  objects  examined  in  the 
SEM  are  intrinsically  3-dimensional.   Polycrystalline  and  biological  materials 
give  rise  to  images  with  gradations  of  tone  and  texture.   Such  images  hold 
a  wealth  of  information  which  at  present  can  not  be  extracted  automatically. 
It  is  an  aim  of  this  proposal  to  rectify  this  situation. 

Operational  facilities  of  the  Illiac  III  Computer  System,  in 
particular  the  S-M-V  Scan/Display  System, will  be  used  in  this  investigation. 
It  is  foreseen  that  image  analysis  is  now  entering  a  phase  of  exuberant 
development  comparable  to  that  recently  seen  in  computer  graphics.   Display 
algorithms  of  this  latter  discipline  are  now  reappearing  turned  inside-out, 
so  to  speak,  as  scene  analysis  algorithms.   It  is  appropriate  therefore  to 
now  make  these  critical  experimental  probes  toward  a  truly  comprehensive 
image  processing  system. 
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